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METHOD AND APPARATUS FOR IMAGE ALIGNMENT 

This invention is concerned with the field of three- 
dimensional imaging and specifically with co-ordinate tracking 
and single-user display applications in which the position of 
the observer is used as feedback to the display system. 

BACKGROUND 

There are various ways of producing a three-dimensional image, 
in which the viewer's left and right eyes observe different 
perspectives of the same subject, thus creating a sensation of 
depth and separation between foreground and background. Some 
systems are based on providing only two perspectives. In these 
cases, it is normal to use special eyewear (such as polarising 
or anaglyphic filters) which transmit the appropriate image to 
each eye. In another type of system, a continuum or plurality 
of perspectives, combined in interdigitated form behind a 
lenticular screen, is projected by this screen, so that in use 
the viewer effectively only observes two of them (one for each 
eye), without requiring special eyewear to do so. The viewer 
may alter his position and observe the subject from a new view 
point. His range of movement is typically limited to some 20 
to 30 degrees around the screen if good image quality is to be 
maintained, and the depth of field is limited by the ability 
of the optical system to separate adjacent perspectives and 
avoid the blurring of them. In a further type of system, the 
aperture (or pupil) of the image projection optics is 
relatively large, and relayed, using suitable optical elements 
to the observer 1 s location. An electro-optic and/or 
electromechanical shutter arrangement, located near the 
aperture, allows different perspectives to be sequenced using 
a single display screen, each perspective only being 
observable at specific locations in space. For real time 
applications , multiple perspective ( autostereoscopic ) 
systems require very high display bandwidth and complex 
display apparatus. 
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There are a variety of different methods by which an 
observer's co-ordinates may be tracked. These include magnetic 
coil based and ultrasonic detection systems. The position of a 
one or more coils or a detector or transmitter is determined 
in three dimensions by measurement of magnitude and phase of a 
plurality of received signals. 



SUMMARY OF THE INVENTION 



It is an object of the invention to provide a method for 
minimising the displayed bandwidth of a three-dimensional 
image * 

It is another object of the invention, in preferred 
embodiments, to provide an auto-stereoscopic image, by 
avoiding the need for special eyewear which would impede the 
viewer's normal vision. 



It is a further object of the invention, to provide parallax 
changes for both horizontal and vertical movement of the 
observer . 



It is yet another object of this invention to provide a simple 
solid state detector for determining the position of an 
observer's head co-ordinates in relation to a location which 
the observer is facing, thereby reducing the headwear worn by 
such an observer to a practical minimum. 



It is yet a further object of the invention to provide a means 
for locating reference points within a determined field of 
view of the apparatus. 



It is also an object of the invention to utilise the infrared 
signals used to control the operation of eyewear for 
stereoscopic viewing, when used by the observer, as a source 
of illumination for head location* 
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Statements of Invention 

Thus, according to the present invention there is provided 
apparatus for the display of autostereoscopic images, 
comprising means for detecting the location of the head of an 
observer with respect to a real and/or virtual image forming 
region; optical means for providing a point or region of 
concentration of light, said light providing in use an image 
of at least a first single perspective of a scene at said 
image forming region, and a void point or region, spaced from 
said point or region of concentration, substantially void of 
light from said first single perspective image; means for 
providing an output signal from the detecting means indicative 
of the observer's head location; and control means 
responsive, in use, to said signal and causing the location of 
the point or region of concentration of light from said first 
single perspective to substantially track and be co-located 
with a first eye of the observer, and said void point or 
region to be co-located with the second eye of the observer. 

According to one aspect of the invention the optical means 
comprises, in combination, a source of light; a lenticular 
screen having a substantially evenly spaced plurality of 
juxtaposed lens elements; an apertured surface having a 
corresponding plurality of light transmitting regions 
separated by opaque regions and positioned close to the 
lenticular screen between said screen and said source of 
light; and means for moving the apertured surface relative to 
said screen, whereby, in use, the control means positions the 
point or region of concentration of light corresponding to the 
perspective image provided by the optical means. 

Preferably diffusing means is provided at the focal plane of 
the lens elements of the lenticular screen. 

The apertured surface may be positioned near the focal plane 
of the lens elements of the lenticular screen and lateral 
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movement of the point or region of concentration of light is 
provided, in use, by movement in a first direction of said 
screen relative to the apertured surface in a direction 
parallel to the plane of said screen. 

The distance from the optical means of the point or region of 
concentration of light may be changed by movement in a second 
direction orthogonal to the first direction of movement. 

Advantageously the optical means comprises a tapered structure 
in which the pitch between adjacent lens elements of the 
lenticular screen reduces in a direction orthogonal to the 
first direction of movement; the transmitting and opaque 
regions of the apertured surface are arranged in a matching 
tapered pattern; and the second direction of movement is 
substantially parallel to the plane of the screen, so that, in 
use, the pitch of the tapered screen, at a given position 
relative to the pitch of the nearest portion of said pattern, 
is altered, whereby the control means controls the convergence 
of the light of the first perspective image and the distance 
of the point or region of concentration thereof from the 
optical means. 

The apertured surface may be positioned substantially at the 
focal plane of the lens elements of the lenticular screen. 

Conveniently, a second optical means provides a second point 
or region of concentration of light from an image of a second 
single perspective of the scene at the image forming region 
and a second void point or region, spaced from said second 
point or region of concentration, substantially void of light 
from said second single perspective image; said second optical 
means includes a semitransparent mirror positioned so that, in 
use, said second perspective image is substantially co-located 
with the first perspective image, one of said images being a 
virtual image; and said second optical means is controlled so 
that, in use, said point or region substantially void of light 
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from said second single perspective image is co-located with 
the observer's first eye and said point or region of 
concentration of light of said second perspective image is co- 
located with the observer's second eye. 

Advantageously, a first transparent image generating surface 
is positioned in the optical path between the means for 
providing the first point or region of concentration for the 
first single perspective image and the observer, and a second 
transparent image generating surface is positioned in the 
optical path between the means for providing the second point 
or region of concentration and the observer, so that both the 
first and second perspective images may be provided 
simultaneously . 

Preferably each image generating surfaces is an LCD panel . 

According to another aspect of the invention a first and a 
second single perspective image of a scene, are projected in 
positional register but at different angles onto the 
apertured surface so that the components of said first image 
transmitted by the apertured surface are projected to 
positions spaced from the corresponding components of said 
second image on the diffusing means. 

According to a further aspect of the invention the optical 
means comprises, in combination, a lenticular screen having a 
substantially evenly spaced plurality of juxtaposed lens 
elements; an image generating surface having a corresponding 
first plurality of spaced columns for providing elongate 
components of the first perspective image of the scene and a 
second plurality of columns, interleaved with said first 
plurality, for providing elongate components of a second 
perspective image of said scene, said surface or an image 
thereof being located at the focal plane of the lens elements 
of the lenticular screen, said screen providing, in use, a 
second point or region of concentration of light, 
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corresponding to said second single perspective image of the 
scene and a second void point or region, spaced from said 
second point or region of concentration, substantially void of 
light from said second single perspective image; and means for 
moving, in its own plane, said image generating surface or an 
image thereof relative to said screen, whereby, in use, the 
control means positions the point or region of concentration 
of light corresponding to each respective single perspective 
image provided by said image generating surface* 

The means for providing movement typically comprises means for 
providing a first direction of relative movement, orthogonal 
to the elongate image components, to control the lateral 
position of each point or region of concentration of light and 
means for providing a second direction of relative movement, 
orthogonal to said first direction, to control the convergence 
of the image light and thereby the distance of each point or 
region of concentration of light from the lenticular screen. 

Advantageously, the optical means comprises a tapered 
structure in which the pitch between adjacent lens elements of 
the lenticular screen reduces in a direction orthogonal to the 
first direction of movement, the elongate components of each 
perspective image are arranged in a matching tapered pattern 
and the second direction of movement is substantially parallel 
to the plane of said screen, so that, in use, the pitch of the 
tapered screen, at a given position, relative to the pitch of 
the nearest portion of said pattern, is altered to control the 
convergence of the light of each perspective image and the 
distance of the point or region of concentration thereof from 
the optical means. 

According to another aspect of the invention the optical means 
comprises a single image generating surface positioned in the 
image forming region; means for selectively providing, by 
transmission or generation, light at different positions on an 
aperture plane spaced from said image generating surface ; 
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convergent optical means controlled by the control means, in 
use, to provide convergence of image light from the image 
forming region, so that said aperture plane is substantially 
conjugate to a plane containing the location of the observer; 
means for alternately generating the first single perspective 
of a scene and a second perspective thereof with a second 
point or region of concentration of light corresponding 
thereto; and means for controlling the light providing means, 
so that in sequence, the points or regions of concentration of 
light corresponding to each single perspective image are co- 
located with each respective eye of the observer. 

According to another aspect of the invention the optical means 
comprises a single image generating surface positioned in the 
image forming region; area programmable means for providing, 
by transmission or generation, light at a plurality of 
discrete locations on a plane spaced from said image 
generating surface; and means for changing the point or region 
of concentration of image light corresponding to a perspective 
image from the image forming region by control of the pattern 
of transmission or generation of light by said programmable 
means to correspond to spaced portions of both the first and a 
second perspective image displayed simultaneously on the image 
generating surface, so that each of the observer's eyes only 
sees respectively the correct perspective image portions. 

Preferably, the separation between light transmitting or 
generating regions of the programmable means and the 
corresponding separation between portions of the same 
perspective image on the image generating surface are 
decreased as the observer's distance from the image forming 
region increases. 

Advantageously, the positions of the light transmitting or 
generating regions of the programmable means are changed over 
a short time interval , so that the observer sees a contiguous 
three-dimensional image comprising light substantially from 
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the whole area of said programmable means. 

According to another aspect of the invention, apparatus for 
detecting the location of a marker and/or the head of an 
observer, comprises a source of infrared radiation; means for 
imaging the space illuminated by said source; and means for 
locating, in use, within the image of said space, the image of 
the marker and/or the head and deriving positional data 
corresponding thereto . 

Preferably, the infrared source is located close to the 
imaging means and the marker is retro-reflective. This may be 
part of the observer's headwear. It may consist of one 
contiguous retro-reflective area. Alternatively, the marker 
may comprise two or more retro-reflective components arranged 
in a defined pattern. 

Advantageously, the imaging means comprises a lens and a CCD 
array. The CCD array may be two-dimensional and provide video 
data. The line scan direction of the video image is, 
conveniently, arranged to be vertical. 

In certain embodiments of the invention each line of video 
data is integrated to provide positional data along a first 
axis . 

Preferably, the infrared source is modulated in synchronism 
with the frame rate of the CCD to provide an output signal 
corresponding to the difference between video data in 
alternate fields. A blocking filter may be used to remove 
light outside the range of wavelengths of interest. 

Following location of the marker's image in one axis, one or 
more lines of video data may be selected to comprise data 
corresponding to the retro-reflective means, whereby a second 
axis of positional data and/or orientation data can be 
derived. The width of the image of the retro-reflector is 
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determined to provide its distance from the detecting means, 
and thus the location of the observer in three dimensions, is 
derived . 

Advantageously, interpolating between intensity values 
associated with two or more discrete locations on the CCD 
array provides more accurate positional data. 

According to a further aspect of the invention each 
perspective image is selected in accordance with the location 
of the observer as provided by the output signal from the 
detecting means. 

In certain embodiments, the pupil of a camera providing an 
image for viewing by the observer is moved in response to 
changes in the output signal, or corresponding positional 
data, to provide movement parallax. 

According to another aspect of the invention a field 
sequential viewing device may be combined with apparatus for 
detecting the location of a marker and/or the head of an 
observer, said apparatus comprising a source of infrared 
radiation; means for imaging the space illuminated by said 
source, in which the imaging means comprises a CCD array 
providing video data; the infrared source is modulated; and 
the frame rate of the CCD and the operation of said field 
sequential viewing device are synchronised to said modulation. 
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DESCRIPTION OF PREFERRED EM B ODIMENTS 

The invention will now be described with reference to Figures 
1 to 13 in which:- 

Figure 1 illustrates a means for detecting the observer's 
position relative to a viewing screen. 

Figure 2 illustrates a two view display system in which the 
points of projection of the displayed views may be altered. 

Figure 3 shows diagrammatical ly a control system for the 
presentation of image data selected according to the 
observer 1 s position. 

Figure 4 shows a form of lenticular screen constructed in 
accordance with the invention. 

Figure 5 shows the light paths in a horizontal section through 
the arrangement of Figure 4. 

Figure 6 shows in diagrammatic form, the aperture segmentation 
of a viewing system constructed in accordance with the 
invention in which the user's head movement is translated into 
movement of the entrance pupils of two video cameras. 

Figure 7 shows an arrangement in which CCD arrays provide head 
tracking data in accordance with the invention. 

Figure 8 shows a display system comprising two conventional 
LCD panels in accordance with the invention. 

Figure 9 shows in plan a schematic view of a system comprising 
in combination a monochrome CRT, a colour selection device for 
field sequential operation and an electro-optic shutter array 
to provide autostereoscopic images in accordance with the 
invention . 

10 
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Figure 10 illustrates a further embodiment of the invention in 
which a shutter array is combined with a display screen to 
provide an auto-stereoscopic display. 

Figure 11 illustrates diagrammatical ly a co-ordinate 

detection system in accordance with the invention. 

Figure 12 is a block schematic of a signal processor 
constructed in accordance with the invention. 

Figure 13 shows diagrammatical ly one aspect of the way in 
which data is processed in accordance with the invention to 
provide co-ordinate data. 

Means for detecting and tracking the location of the head of 
an observer is shown in Figure 1. The observer 1 is located in 
front of a display screen 2 (shown as a broken line). The 
detection and tracking system is located immediately above 
screen 2. The observer is provided with a small retro- 
reflective patch or marker 3, which is worn centrally on the 
forehead, being the only headwear required. An infrared light 
emitting diode 4 immediately behind a semitransparent mirror 5 
emits a wide beam of radiation 6 towards the observer. The 
retro-reflecting patch 3 returns a much narrower beam 7 to 
mirror 5. The returned radiation reflected by mirror 5 allows 
the formation of an image 13 of patch 3 by a lens 8 onto a 
split photodetector 9R and 9L which portions detect the right 
and left portions respectively of the image. Mirror 5 is 
mounted on the vertical shaft of motor 10 (drawn with a dashed 
line for clarity). As the observer moves laterally, the signal 
on one half of the detector changes relative to that on the 
other half. The difference between the two signals is a 
measure of the distance by which the observer's head location 
has changed, as the position of image 13 moves accordingly. By 
rotating mirror 5, this output or error signal can be nulled 
as part of a closed loop control system, the angular position 
of the mirror then being a good measure of the observer's 

11 



BN8DOCID: <WO_jM20875A2JU> 



WO 94/20875 



PCT/GB94/00405 



position. At any given moment, the angular position of the 
mirror and/or the relative strength of the signals from left 
and right detector segments provide a measure of the 
observer's location. 

Optical means, in the form of a display screen for producing 
two spatially separated but controllable viewing locations, 
each with a different associated image, is illustrated with 
the aid of Figure 2. This is not drawn to scale for the sake 
of clarity. The observer's eyes, 11R and 11L are located at a 
distance D in front of a hybrid screen, consisting of a 
lenticular screen 12 (normally used for multi-perspective 
images and having an evenly spaced plurality of juxtaposed 
cylindrical lens elements formed on its front surface) of 
thickness t and refractive index n placed a distance s in 
front of a flat display surface 13 with columns of picture 
elements. The columns are arranged to match the pitch between 
the lenses of screen 12 and alternate ones R and L are seen 
respectively by the observer's right and left eyes. 
Preferably, the focal plane of the lens elements of screen 12 
corresponds to the position of surface 13. Two viewing regions 
(or regions of concentration of image light from each 
perspective) 14R and 14L, corresponding to the R and L columns 
respectively, are created as a result of the imaging 
properties of screen 12. Providing the vertical plane 15 
dividing regions 14R and 14L remains between the observer's 
eyes, the latter will see a stereoscopic image. The angle, 
shown as near zero for a centrally positioned observer, which 
plane 15 makes with respect to the normal to screen 12, in 
practice, is controlled by moving the latter, horizontally in 
its own plane, relative to display surface 13. In practice, a 
very small movement 16 of screen 12 produces a substantial 
movement 17 at the observer's position of plane 15. To a good 
approximation the ratio of these movements R is given by the 
equation: - 

R = D / ( t/n + s ) 
12 
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Using typical values (distances in millimetres) for the 
parameters involved, D = 600, t = 3, n = 1.5 and s = 1, a 
value for R of 200 is obtained. In this example if the head 
tracking apparatus of Figure 1 detects a lateral head movement 
of 200 mms, a lateral movement of 1 mm of screen 12 relative 
to the display surface 13 will be sufficient to maintain 
alignment of the stereoscopic image with the observer. The 
movement of screen 12 may be effected in a variety of ways, 
which are not explicitly illustrated. It will be clear that a 
very simple and direct method is to provide a suitably geared 
linkage with the rotation of the shaft of motor 10 (Figure 1). 
The control means is made to respond to the difference between 
the outputs of diodes 9R and 9L and thereby to position the 
viewing zones correctly for stereoscopic viewing. 

One particular embodiment of apparatus for locating the 
correct viewing zones of a stereoscopic image correctly for an 
observer has been illustrated. Other embodiments in accordance 
with the invention which achieve the same result are quite 
practical . One such alternative involves the use of two 
display screens, one for the left eye image and the other for 
the right eye image. The two screens are positioned at 
equivalent locations either side of a semitransparent mirror, 
one being seen as a virtual image in the image forming region. 
Each screen produces a localised viewing zone for its 
respective eye, which can be achieved in a manner similar to 
the two view system described above. Each screen is therefor 
only visible to one of the observer f s eyes, where there is a 
point or region of concentration of the image light 
corresponding thereto, and the space adjacent to this point of 
concentration, where the observer's other eye is positioned, 
is void of light from that perspective view. In other words, 
the point or region of concentration of light from one 
perspective view is co-located with the point void of light 
corresponding to the other perspective view. In this way, the 
observer sees a stereoscopic image. Such an embodiment is 
described below with reference to Figure 8. In this 
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description the phrase "point or region of concentration of 
light" is used principally to convey the relative strength of 
light at such points or in such regions, when compared with 
that at corresponding points or regions substantially void of 
such light, and need not imply the application of a process by 
which luminous flux is increased in an absolute sense. 

Another method of correctly locating the viewing zones at the 
observer's position is to place a mirror intermediate the two 
view display system and the observer. By rotating this instead 
of laterally moving a lenticular screen as described above, 
the points or regions of concentration of image light from the 
left and right eye perspective views of a scene (viewing 
zones) can also be made to track the eyes of the observer, in 
order to maintain good stereoscopic viewing. 

A system for presenting stereoscopic subject matter, the 
perspective view of which is driven by the observer's 
position, is shown diagrammatical ly in Figure 3. In this 
example the photo-detector 18 is a four segment device which 
provides difference signals for vertical and horizontal motion 
detection by summing the outputs of the quadrants in pairs, 
either horizontally to provide a differential signal pair for 
vertical movement, or vice versa. An additional mirror and 
motor drive 19 is provided to null the vertical error signal 
and thus track the observer's vertical position. This is not 
necessary in order to correctly position the two required 
viewing zones, but does provide the second axis for tracking 
observer position. Controller 20 receives 4 quadrant detector 
inputs 21 and provides the control for the angle of the two 
tracking mirrors and the horizontal actuator 22 for the 
lenticular screen 23. A further output 24 provides a digital 
representation of the observer's location to a graphics 
controller 25. This selects the appropriate views for 
presentation by the display driver 26 on the flat display 23A. 
In this manner movement parallax may be provided to the 
observer with a display bandwidth which is not excessive. 

14 
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The embodiment of Figure 2, in which a lenticular screen 12 is 
used to project left and right eye views generated in 
conjunction with a matching segmented display surface 13, 
suffers one limitation, when a conventional lenticular screen 
is used. This limitation is, for a given ratio (less than 1) 
of the lenticular spacing to that between alternate display 
surface segments, that the observer must be positioned within 
a narrowly defined region a certain distance from the screen. 

In Figure 4, a form of lenticular screen 27 which removes this 
restriction and is constructed in accordance with this 
invention is shown. Its essential feature is that it has a 
tapered structure. The display screen's segmentation has a 
matched taper. The lines 28, shown as dashed lines behind the 
lenticular screen represent the boundaries between left SL and 
right SR eye segments of the displayed image. The display 
screen comprises a first and second plurality of columns, each 
plurality providing elongate image components of a respective 
perspective view. In use, the lenticular screen is moved 
vertically 29 in its own plane relative to the display 
screen's segmentation pattern, which is at the focal plane of 
the lenticular screen's lens elements. This selects, for each 
element of the displayed image, the appropriate ratio between 
its segment spacing and that of the lens elements, at the 
nearest point of the lenticular screen. Such vertical 
adjustment may be controlled manually to allow the user to 
select a convenient viewing distance. Alternatively, the 
observer's distance may be determined by a head tracking 
arrangement as described with reference to Figures 7 or 10 to 
11 below, or an alternative method, and the screen's position 
control means automated. 

The geometric limitations imposed by the condition that good 
autostereoscopic viewing be achieved across the entire width 
of the displayed image is illustrated in Figure 5. The 
lenticular screen pitch and the display screen's segmentation 
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should have a specific ratio for any given viewing distance. 
Lines 30 represent the projections of two of the boundaries 
defined by lines 28 in Figure 4. There are restricted regions 
of perspective light concentration, VL and VR, at which 
correct stereoscopic viewing is achieved. Assuming an 
interocular spacing of 65 nuns and the values used previously 
for the lenticular screen's effective focal length of 3 rams, 
but varying the viewing distance, the ratio is 0.994 at 
500 mms, whereas at 1000 mms it becomes 0.997. If this ratio 
is not changed, the operating range R for a screen of up to 
400 mms width at a viewing distance of 500 mms is, under 
optimum conditions, +/- 80 mms, but, allowing in practice for 
some latitude in left and right eye viewing location 
separation and lateral positioning, may be as low as 
+/- 54 mms and that at the 1000 mms distance would, 
correspondingly, be + /- 85 mms. 

This invention can be applied in many fields. These include 
both remote handling and virtual reality applications. A 
particular, form of remote handling occurs in the medical 
field where closed circuit video cameras are used as a tool in 
so-called "Key hole" or minimally invasive surgery. The 
surgeon must operate with a video image from a laparoscope in 
order to see what he is doing. Availability of the third 
dimension is recognised as a key requirement in order to 
reduce the risk of errors and to make procedures such as 
suturing an easier task. 

In Patent Application PCT/GB_92 /00292 (Street), the principle 
of segmentation of a large pupil (lens aperture) into smaller 
regions, each of which provides a slightly different 
perspective of the subject matter, is described. One pupil 
segmentation arrangement for a stereoscopic vision system 
providing some movement parallax for a surgeon whilst 
performing a surgical procedure is shown diagrammatical ly in 
Figure 6. The exit pupil of the laparoscope is shown as a 
broken circle 31. The diameter of this pupil is typically 
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4 mms . At or close to puoil 31 there are two aperturing 
devices. A narrow horizontal slot is formed by the separation 
of plates 32 and 33, and two shaped cut-outs 34 and 35 are 
provided in circular plate 36. A rotation A' of plate 36 
causes the distance between the two transmitting regions 37 
and 38 of the arrangement (hatched for clarity) to change • The 
two video cameras (not shown) collect light respectively from 
regions to the left and the right of boundary line 39 which 
passes through the centre of disc 36. The whole aperturing 
arrangement of plates 32 , 33 and 36 may be moved 40 relative 
to pupil 31. In an autostereoscopic system constructed in 
accordance with this invention, the head tracking data is 
used, both to ensure that the displayed pair of images is 
projected correctly to the observer's left and right eyes 
respectively and also to control the rotation of plate 36 for 
distance from the viewing screen and, for small and/or rapid 
head movements, the co-ordinates of the transmitting regions 
37 and 38 with respect to pupil 31. In this way the effective 
pupil position of each camera is moved to provide some 
movement parallax for the observer. 

A head tracking detector arrangement providing lateral , 
vertical and distance data requiring no moving parts and using 
a retro-reflective patch worn by the user is illustrated in 
Figure 7. The reflector 41 is worn by the user 42 as part of 
his normal headwear. Light from an infrared emitter 43 is 
directed towards the user via a semit ransparent mirror 44. The 
returned light 45 is imaged by lens 46 after passing through 
mirror 44. Half of the imaged radiation is directed by beam- 
splitter 47 through a cylinder lens 48 onto a vertically 
oriented linear CCD array 49. The remainder forms an image on 
the horizontally oriented CCD array 50. A second cylinder lens 
51 concentrates the infrared light from lens 46 vertically, in 
the same way that lens 48 performs this function horizontally 
for array 49. When each CCD is read out in the conventional 
way, the respective horizontal and vertical location of the 
images on arrays 50 and 49 are found. The output of CCD 50 is 
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shown as trace 52, Voltage V is plotted against position X 
along the CCD. The two edges of the wave form correspond to 
the positions of the two edges of the image of patch 41 formed 
on the CCD. The greater the distance of the user from the head 
tracker, the smaller the gap between the rising and falling 
edges. From these two edges, both lateral position and 
distance may therefor be derived • 

Two linear CCD arrays to provide horizontal and vertical 
position data have been described. A single two-dimensional 
CCD and a lens, as used in many video cameras, may be employed 
to image the space illuminated by the infrared radiation to 
provide both co-ordinates of the retro-reflector. By 
synchronised modulation of the infrared diode, alternate 
fields of the video signal may comprise the scene with and 
without infrared illumination. By subtraction, only the image 
of the retro-reflector remains for subsequent position 
analysis, whilst the scene comprises many irrelevant features. 
Such an embodiment is described below with reference to 
Figures 11 to 13. 

The embodiments described in some detail, so far, have 
illustrated the use of one image bearing or generating surface 
or screen comprising both left and right eye image segments. 
Each has typically been made visible to the observer's 
appropriate eye by employing a lenticular screen interposed 
between the observer and the surface comprising the image 
segments. A system comprising two separate and, in principle, 
conventional liquid crystal display or LCD panels is described 
with with reference to Figure 8. Two equivalent optical means 
comprising lenticular structures serve only to direct the 
light projected through each panel, or transparent image 
generating surface, to the observer's corresponding eye, by 
doing this from behind the image forming elements of each LCD. 

Light from source 53 is collected by lens assembly 54 and 
split in two parts by beam splitter 55. For efficiency this 
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may, optionally (not illustrated), be a polarising beam 
splitter and the axes of polarisation may be aligned to those 
of the input polarisers of the subsequently encountered LCD 
panels. Light passes via mirrors 56 and 57 to the rear of 
barrier strip screens 58 and 59 respectively. These typically 
comprise an apertured surface with alternate transmitting and 
opaque stripes running vertically with respect to the image to 
be displayed. Each such barrier strip is positioned close to 
and at a precisely controlled distance from a diffusing 
surface at the focal plane of a corresponding lenticular 
screen. Lenticular screen 60 is in front of barrier strip 58 
and lenticular screen 61 is in front of- barrier strip 59. For 
a plurality of lenses on a lenticular screen there is a 
corresponding plurality of transmitting and opaque regions on 
its associated barrier strip screen. 

Means is provided to move each lenticular and corresponding 
apertured surface relative to one another in two directions. 
The lateral position of each lenticular screen relative to its 
corresponding barrier strip is controlled by cam arrangements 
64 and 65 respectively. This first direction of relative 
movement is in the plane of the lenticular screen. The cams 
are schematically drawn and are countered by return springs 
(not shown). The distance between each barrier strip and the 
diffuser of its corresponding lenticular is controlled by cam 
arrangements 62 and 63 respectively f which provide a second 
direction of relative movement, orthogonal to the first. The 
distance between centres of the transmitting regions of the 
barrier strip screen is a close match to the distance between 
centres of the lenticular lens elements. By small lateral 
movements in the first direction of the lenticular with 
respect to the striped image formed on its corresponding 
diffuser by light passing through the barrier screen from the 
compact light source 53, the direction of the light leaving 
the front of the lenticular is controlled to arrive in the 
region of the observer's appropriate eye, after passage 
through the corresponding LCD panel. The scale of the striped 
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image may be altered slightly to achieve the correct degree of 
convergence of the projected light to match the observer's 
distance from the display surface. This is achieved by 
altering the separation of each barrier strip from the 
corresponding diffusing surface. As the light from source 53 
is diverging, a greater separation results in a larger scale 
and corresponds to greater convergence of projected light, 
which moves the concentration of the image light (and the 
points void of such light) towards the screen providing the 
observer with a closer viewing point. The two LCD panels 66 
and 67 are both observed in the same location, or image 
forming region, with the aid of a semitransparent mirror 68 , 
but by different eyes. The observer's left eye 69 sees light 
arriving from panel 67* The right eye 70 receives 
substantially no light from this panel as the light blocking 
regions of barrier screen 59 prevent this, forming a region 
void of light from the left eye perspective image formed on 
panel 67. Conversely, the observer's right eye 70 does see the 
virtual image of panel 66 and the left eye 69 does not. Panels 
66 and 67 are fed video signals on inputs 71 and 72 
respectively. Provided these correspond to the left and right 
eye images required, good depth perception is achieved. 

A variant of the embodiment of Figure 8 may be constructed in 
which both the lenticular and barrier strip screens have a 
tapered structure, in which the pitch between lens elements 
reduces in a direction orthogonal to the first direction of 
relative movement and each apertured surface has a matching 
pattern of alternating transmitting (transparent) and opaque 
stripes (regions). Such a structure has already been described 
with the aid of Figures 4 and 5. In this case the diffuser 
may be at the plane of the barrier strip, which in turn can be 
at the focal plane of its corresponding lenticular screen and 
is moved in its own plane relative thereto and vertically with 
respect to the image, in order to change scale. A diffuse 
source of light such as a conventional back light may then be 
used instead of a compact source. 
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The embodiments of Figures 2 to 5 and Figure 8 have 
illustrated apparatus in which left and right eye images are 
displayed simultaneously. It is also possible to embody the 
head tracking principles of this invention in a field 
sequential display incorporating a single contiguous image 
forming surface, at which both left and right eye perspectives 
are generated. 

In European Patent Specification No 0118449 (Street) an 
imaging system is described in which a three-dimensional 
object space is imaged through a wide aperture to a spaced 
conjugate image forming region where a three-dimensional image 
may be observed and recorded. One embodiment of that invention 
provides for the sequential generation of a series of two- 
dimensional perspectives of a three-dimensional object by an 
electronic imaging device, such as the face of a Cathode Ray 
Tube (CRT). By imaging each of these perspectives to the image 
forming region through a respective series of adjacent 
apertures, a three-dimensional image is formed, over a time 
interval, and may thus be recorded. If the rate of sequencing 
this series of perspectives is fast enough and repeated say 50 
times per second, such a three-dimensional image would be 
viewable in real time without substantial image flicker. 

The current state-of-the-art allows for 400 frames to be 
provided every second, which provides the opportunity to 
display a three-dimensional image of reasonable resolution 
comprising 16 perspective views at a repetition rate of 25 
cycles per second. For reasons of image data rate, light 
efficiency and phosphor decay time, there are difficulties in 
providing colour at these rates. If field sequential 
generation of colour is used, with, for example, an LCD colour 
shutter of the type manufactured by Tektronix Inc. under the 
brand name NuCOLOR™, a loss of a factor of six in light 
efficiency may be anticipated when working with polarised 
light. As the number of perspective views is increased, the 
amount of light transmitted by the system in unit time reduces 
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in a reciprocal manner. For a seamless three-dimensional image 
to be observed, one in which there is no perceptible stutter 
between adjacent perspectives, it is necessary to provide a 
new view of the subject matter at 2 mm intervals at the 
viewer's location. At a typical distance of 1 meter from the 
image forming region, this represents a resolution between 
adjacent perspectives of 0.115°, Thus, for a totally efficient 
system, in which both perspective and colour are sequenced and 
a viewing angle of 30° is to be achieved, the maximum light 
throughput would be 0.032% of the light available at the image 
forming surface of the CRT. In practice, the data rate would 
be unachievable anyway and a tolerable light efficiency would 
need to be in excess of 1%, given a very high brightness CRT. 

The current invention may be used to increase perspective 
angle resolution and viewing angle, whilst both retaining a 
relatively high duty cycle in the viewing locations required 
by the observer and containing the image data bandwidth 
required. Figure 9 illustrates such an arrangement of 
components schematically. A CRT 73 generates a sequence of 
two-dimensional images or perspective views. A typical image 
point P on the CRT's spectrally broad phosphor coated surface 
emits light in Lambertian fashion. Lens components 74, 
providing the function normally required for TV projection, 
collimate rays from point P. These rays pass through a colour 
shutter 75 which selects red, green and blue image components 
in rapid succession. The rays transmitted by shutter 75 
encounter an electro-optic shutter array 76. The function of 
array 76 is to provide light for viewing by selective 
transmission thereof at a plane spaced from the image 
generating surface, acting as an aperture plane. (In 
embodiments in which this light providing component is behind 
a transparent image generating surface, it may fulfil the same 
function by being a light generating surface, such as the face 
of a CRT.) Array 76 is shown to have two open regions 77 and 
78. In practice these are opened sequentially and are not open 
simultaneously. Lens components 79 refocus the transmitted 
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rays from object point P *-.o form an observable image point P f 
at the rear surface of lens assembly 80. This comprises a 
Fresnel component 81 and a varifocal surface 82 , shown as a 
pair of dashed lines at its two extreme radii of curvature. 
The function of the Fresnel is to image the shutter array 76 
to the average distance at which the viewer will be 
positioned, whereas the power contributed by the surface 82 
will allow this to be adjusted, providing the convergent 
optical means to vary the convergence of image light, so that 
the plane of the shutter array 76 or aperture plane may be 
made conjugate to the plane containing the location of the 
observer. When region 77 of shutter 76 transmits light, the 
viewer's right eye 83 sees the image of the CRT 73 reformed at 
Fresnel lens 81. When region 78 is enabled, the left eye 84 
will observe the image. A fully autostereoscopic image in 
colour can be produced in this way by alternately generating 
left and right eye images with 6 frames of data (3 colours x 2 
perspectives). The resolution of the shutter array 76 can be 
high and thus the tracking of the observer's head location can 
be as accurate as the head tracking subsystem already 
described allows, without any increase in image data 
bandwidth. In more sophisticated embodiments of the invention 
and at the expense of light efficiency, each transmitting 
region of the shutter array may itself be subdivided into 
closely spaced perspective viewing regions with the CRT having 
to display more frames within the total cycle time available. 
In this way a system with virtually no image lag where image 
data is driven by the user's head position, for example in 
Virtual Reality applications, may be constructed. As in 
previously illustrated embodiments, data provided by the head 
tracking subsystem 85 is used to control both the lateral 
location of viewing locations, by selecting the corresponding 
regions of array 76, and, through a controllable reservoir 86 
containing optically clear fluid and actuator 87, the power of 
surface 82. This, in turn determines the distance from Fresnel 
lens 81 at which the shutter array is re-imaged. The result is 
that the plane of array 76 is conjugate to the plane of the 



23 



BNSDOdD: <WO_042O875A2JU> 



WO 94/20875 



PCT7GB94/00405 



observer, providing correct stereoscopic viewing over the 
whole image area. The sequential transmission of image light 
at points 77 and 78 provides sequentially the two points or 
regions of concentration of image light required by the 
observer's eyes 83 and 84 respectively* 

In Figure 10 f a simple auto-stereoscopic display constructed 
in accordance with this invention is illustrated. This uses a 
conventional two-dimensional flat CRT screen, monochrome or 
colour, as the image generating surface, in front of which, in 
this example spaced by 100 nuns , there is a ferro-electric 
(electro-optic) shutter array 88 similar to that of Figure 9* 
There are no other components required in the optical path and 
the image plane or generating surface could equally well be 
provided by projection. The screen 89 simultaneously displays 
alternate vertical bands or spaced portions of the two 
perspective views to be seen at the left eye location 90 and 
right eye location 91 respectively of the observer (shown 
schematically). The shutter array 88 is an area programmable 
device, programmed to have vertically arranged slit like 
apertures 92 at equal intervals S x • In the embodiment 
illustrated it must provide light by transmission at this 
plurality of spaced discrete locations. A convenient width for 
each aperture is typically one quarter of this spacing. The 
head tracker module 93 provides the observer's co-ordinate 
data to controller 94 and this, in turn arranges the image 
bands 95 (hatched) of the left perspective view to be 
positioned relative to apertures 92 so that they are only 
visible to the observer 1 s left eye 90 (rays 97), whilst bands 
96 of the right perspective view are seen by the observer's 
right eye 91 (rays 98). The positions and width of apertures 
92 themselves are also controlled by controller 94. The whole 
pattern is shifted rapidly between four locations on a pitch 
of S x /4. The top left hand corner of the pattern at each 
position is marked respectively as 99A, 99B, 99C and 99D. Over 
a short time interval, each part of the entire area of array 
88 transmits for up to 25% of the time available. For each new 
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arrangement of apertures 92, the selection of image data in, 
and the positions of, bands 95 and 96 are rearranged in the 
appropriate pattern. The result is that the observer is 
presented with a contiguous three-dimensional image without 
the need to wear special eyewear. In fact, shutter array 88 
provides the necessary light blocking function. 

The observer's distance V from screen 89 and the spacing D, 
between array 88 and screen 89, determine the correct values 
for S x and the related spacing S 2 between the closest but 
spaced bands of image data for each eye. It will be apparent, 
that, for any finite viewing distance, S 2 must be larger than 
Si in order to achieve convergence of image light, rays 97 
from the left, and rays 98 from the right eye image, towards 
each respective eye location, 90 and 91, of the observer. 
There will then be a region void of light, corresponding to 
one perspective, in locations of concentration of image light, 
corresponding to the other perspective. For correct 
convergence, the following relationship applies: - 



x V 



(V - D) 



Additionally, the magnitude of S x and thus S 2 must be set to 
ensure that each image band is of the correct width, so that 
it does not interfere with either the band corresponding to 
the other eye, as seen through the same aperture, or with that 
corresponding to the adjacent aperture. Examples of practical 
values for the 25% duty cycle arrangement illustrated here 
are as follows:- 

V = 500 nuns S* = 32 mms S 2 = 40.0 mms 

V = 1000 mms Si = 16 mms S 2 = 17.8 mms 

V = 1500 mms S x = 8 mms S 2 = 8.6 mms 

As the observer's distance increases then, given a fixed eye 
spacing of say 65 mms, the values of Si and S 2 must be 
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decreased. In practice, controller 94 arranges the pattern of 
transmission of array 88 to control both the distance from the 
display and the lateral position of the required points of 
concentration of image light of both perspective views for 
correct auto-stereoscopic viewing. The lateral position is 
simply controlled by adjusting the relative phase in space at 
the centre of the image between the pattern of image data and 
the transmission pattern in front of it. 

Given that the screen 89 can be refreshed at a rate of not 
less than 100 full frames per second, then the complete image 
is produced 25 times per second. Higher screen refresh rates 
are possible, but at the expense of image resolution. In 
certain cases the phosphor decay time and the time of the 
frame, which conventionally progresses from the top of the 
image to the bottom, impose awkward timing constraints on the 
operation of shutter 88. This limitation can be mitigated by 
subdivision of the shutter array in the vertical direction* 
Segments at different vertical positions may then be operated, 
at different times, and remain in phase with the progressive 
generation and decay of the image. Another alternative is to 
use a screen with a vertical line scan. Here different 
apertures are opened, at different times, to remain in phase 
with the progressive generation of the image from one side of 
the screen to the other. Another advantage of this arrangement 
is that the bands of image data corresponding to each eye are 
composed of complete lines of such data, instead of the 
arrangement, in which each horizontal line must contain data 
from both perspective views. 

In Figure 11, a further embodiment of a head location system 
is illustrated. Imaging means is provided by a two-dimensional 
CCD array 101, with its read out (scan line) direction 
arranged vertically, located behind an image forming lens 102. 
Infrared light or radiation is emitted, in the form of a cone 
103 of radiant energy, by a ring of light emitting diodes 
( LED ' s ) 104A to 104F grouped around lens 102. By positioning 
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the LED's close to lens 102 the latter receives a significant 
proportion of the light returned a retro-reflective marker 105 
located within circle 106, a section through the cone of light 
103 at a distance Z from lens 102. The shaded square area 
within circle 106 corresponds to the shaded portion of CCD 
101, where an image 107 of marker 105 is formed. Unwanted 
radiation from other sources is reduced by filter 108 which 
blocks visible light. 

In use, the co-ordinates of marker 105, worn as part of the 
headwear of an observer, are determined by analysis and 
location of the image 107 formed on CCD 101. Up to four 
degrees of freedom (two positional, one rotation and scale) of 
marker 105 may be determined in this way, provided the size 
and shape of the marker are known* The width w of image 107 is 
ratioed to the known width M of marker 105. The latter' s 
distance Z from lens 102 is given to first order by the 
following relationship. 

Z = F(M + w) / w 

where F is the focal length of lens 102. In practice allowance 
is made for any image distortions of the camera lens which may 
require the addition of terms in x and y, where x and y are 
the co-ordinates of the centre of image 107 with respect to 
the centre of the shaded area of CCD 101. 

The X and Y co-ordinates of the centre of marker 105 with 
respect to the centre of the shaded area within circle 106 are 
determined by similar relationships as follows. 

X = ( M / w ) x 

Y = ( M / w ) y 

A fourth degree of freedom for an observer is rotation (or 
roll) of the head about the line of sight. This causes the 
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image of the marker to be skewed on the CCD array. The skew 
angle A" can be derived as described below with the aid of 
Figure 13. 

Figure 12 illustrates a typical processing system constructed 
in accordance with the invention. The infrared light source is 
modulated in order to extract from the CCD the basic data 
required to calculate the X, Y and Z co-ordinates of the 
observer 1 s marker . 

The CCD 109 is shown schematically together with the image 110 
of marker 111 within its frame. Block 112 represents the 
group of LED 1 s which emit infrared radiation 113 towards 
marker 111. Returned radiation 114 forms image 110 through 
lens 115. The group of LED 1 s 112 is controlled by line 116 
from control circuit 117 • By synchronising the switching of 
the LED ' s to the frame sync received from the CCD on line 118, 
these are turned on and off during alternate video fields. 

In order to establish the horizontal position of the marker's 
image simply and quickly, the video data is processed by an 
integrator 119 on a line by line basis in real time, providing 
data for analysis along a first (horizontal) axis. A voltage 
corresponding to the total light received on each vertical 
line is produced and, for the field when the LED 1 s are 
switched off, the values for all lines are stored in a buffer 
120. The flow of data is controlled by switch 121, in turn 
driven by control circuit 117. The light level for each line, 
when the LED 1 s 112 are on, is compared with the corresponding 
value in the previous field when the LED ' s were off. 
Comparator 122 performs this simple task and the resultant 
difference values are processed by a small microprocessor 123. 
In alternative embodiments the intermediate storage and 
comparison functions may both be handled within the processor. 

By analysis of the integrated and normalised light values for 
each vertical line of the CCD, the centre x and width w of the 
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marker's image may be determined. These yield respectively the 
values of X and Z using the relationships already provided. 

Figure 13 illustrates how the integrated signals used for the 
determination of x and w may be used to select one or more 
lines of raw video signal and how, from these, the values of y 
and A" may be provided. CCD 124 has an image 125 of the marker 
which is inclined at an angle A n to the horizontal* The 
integrated values of the video signal's vertical scan lines 
are shown along the horizontal axis as trace . The raw video 
data is extracted for two lines at respective x co-ordinates 
xl and x2, both of which are within but near the ends of the 
marker's image formed on the CCD. These are shown as traces 
Vyx and V y2 respectively. Each trace, which is processed in 
order to remove background biases using alternately 
illuminated fields in the same way as for the line integrated 
data, provides a signal which, when analysed, yields centre 
points for the marker's image. For traces V yl and V y2 the 
respective centre points Yl and Y2 are as follows 

Yl = ( Yll + Y12 ) / 2 and Y2 = ( Y21 + Y22 ) / 2 

Thus, along the second axis of positional data, the vertical 
position y is given by 

y = ( Yl + Y2 ) / 2 

The orientation of the marker, angle A", is derived by ratios 
of differences as follows. 

tan ( A" > = ( Y2 - Yl ) / ( X2 - XI ) 

These operations may be handled within a microprocessor or in 
special purpose hardware. When a microprocessor is used, the 
centre of the image may be located to best precision by using 
interpolation. A CCD is inherently digital in the spatial 
domain, that is to say, it usually has discrete photosites 
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(pixels) positioned between 10 and 20 microns apart. With an 
image forming lens of 5 mms focal length and a 20 micron pixel 
spacing this translates to a 4 mms quantisation of the co- 
ordinate system at 1 metre from the lens, if only discrete 
pixel positions are allowed. This is the result if a simple 
threshold is used to determine whether an image is present or 
absent at any given photosite. A regime which provides greater 
accuracy uses the analog image intensity values at more than 
one pixel position to infer the location of the edge of the 
image of the marker to sub-pixel spacing accuracy by 
interpolating between these values. A simple algorithm is to 
assume that image intensity changes in a linear manner 
between two values which lie either side of the chosen 
threshold level. 

Thus, if V^, the value at one pixel location, lies below the 
threshold V T and V K r the value at an adjacent pixel location, 
is greater than Vrr/ the linearly interpolated position p of 
the image edge may be derived as follows. 

p = ( V T - Vx. ) / ( V„ - ) 

where p is less than unity and represents the distance of the 
image edge as a fractional pixel spacing from the pixel with 
the lower value in the direction of the pixel of higher value. 
Not only does this regime produce a more accurate co-ordinate 
determination, it also avoids edge jitter due to noise on a 
value which lies close to the chosen threshold. 

Specific embodiments of the invention have been described. 
There are a number of other ways of realising the invention in 
practice. The marker need not comprise a single contiguous 
retro-reflecting area as illustrated, but may consist of two 
or more reflectors positioned in a geometrically precise 
manner relative to one another. Reflective tape or discreet 
cat's eye reflectors may be used. The illustrated embodiments 
show how the position of one retro-reflective marker may be 
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determined within the ^ield of view of a video camera 
sensitive in the infrared- Other implementations are possible 
in which the radiation used comprises different wavelengths 
and different markers preferentially return respectively 
different wavelengths. In this way the location of different 
markers may be determined. Alternatively, by using more 
sophisticated image processing a plurality of markers may be 
located within one image received over the same spectral 
range, A fixed set of markers may be used to calibrate the 
transformation from image co-ordinates to space co-ordinates. 
Whilst the use of one camera has been described, two or more 
may be used simultaneously to extend the field of view or to 
triangulate the position of a marker to very high precision. 

Whilst the embodiments described have not required the use of 
special glasses to view a stereoscopic image, such eyewear may 
be combined with the infrared head tracking principles 
described herein. When electro-optical ly shuttered eyewear 
controlled by frame synchronised infrared signals is used to 
provide different left and right eye images, the same infrared 
source can provide the required illumination for the retro- 
reflective marker . 

A variety of different display technologies may be used in 
accordance with the current invention. The display surface 13 
of Figure 2 may be a specially constructed LCD panel 
preferably with back lighting. The surface may comprise light 
emitting elements such as a light emitting diode or polymer 
array. Surface 13 may be one on which a first perspective view 
and a second perspective view of a scene are projected from 
one or more video projectors located elsewhere. In this case 
it will be an image of the image generating surface which is 
focused at the focal plane of the lenticular screen. Two 
images may be projected through an apertured surface or 
barrier strip screen, comprising substantially vertically 
oriented stripes, alternately clear and opaque and at a pitch 
compatible with that of the lenticular screen. The structure 
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may be tapered as previously described. This apertured surface 
is placed close to surface 13 which has diffusing properties, 
between it and the image projectors. By maintaining accurate 
register between, but choosing different horizontal angles of 
projection for, the two images, each results in a series of 
spaced components which become interdigitated on surface 13 in 
the necessary way to match the pitch of the lenticular screen. 
The lenticular screen 12 and/or the barrier strip screen are 
moved to track the observer. A first direction of movement , 
orthogonal to the elongate elements of the interdigitated 
image provides left to right control of the viewing point, 
whereas, convergence of the image light is changed by movement 
orthogonal to this first direction. Alternatively, the 
displayed image data may be moved relative to the screen. The 
means for separating the perspectives at the observer's 
location need not be lenticular. As already illustrated, other 
arrangements such as a barrier strip or aperture segmentation 
may be used. The principles of this invention may be combined 
with those described in US Pat. No. 4,641,178 (Street) in which 
two perspectives of a scene are presented within the bandwidth 
of a monoscopic colour display, the spectral content of each 
perspective complementing that of the other at a given 
instant. By switching this spectral division between 
perspectives at field rates, an essentially flicker free 
stereoscopic full colour image is achieved using a 
conventional display bandwidth. In one embodiment the 
switching of spectral components between the observer's two 
eye positions is achieved using an oscillating lenticular 
structure. By super-imposing a suitable bias, in accordance 
with the current invention, on the average location of such a 
vibrating lenticular structure, the observer's head location 
would also be tracked. 

In Figure 10 a particular arrangement is described in which 
the display screen 89 is positioned behind an area 
programmable shutter array 88. Typically the display screen 
will be a CRT or other light emitting image generating 
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surface, such as in an electro-luminescent display'.- or light 
emitting polymer. The display screen may, however, be a 
spatial light modulator such as an LCD panel, providing this 
is fast enough as it would be in the case of a ferro-electric 
LCD. A separate light source would then be positioned behind 
the transmitting display panel. When the image generating 
surface is transparent, it may be placed in front of the area 
programmable device. Whilst this may be a shutter array in 
front of a separate light source, it can conveniently comprise 
a single high brightness CRT, which then becomes an area 
programmable means for generating light, using the principles 
of the current invention. In this case Sj. must be larger than 
S 2 in order to achieve the correct convergence for the 
observer . A negative value for D in the formula relating these 
two values achieves this result. Where an embodiment comprises 
"a single image generating surface", this does not preclude 
multiple surfaces for generating the image. These may be in 
series (transmission) , side by side or combined in other ways. 

Head tracking for autostereoscopic vision may be achieved in 
different ways. An alternative to using a retro-reflective 
patch worn by the observer, is to provide an infrared or other 
light source as part of the observer's head gear. An image of 
this source would be formed in the way already described. With 
more sophisticated image processing, the image of the 
observer's face could be tracked without any special headwear. 
Alternatively, other active sources may be used, with suitable 
detectors. For instance, an ultra-sonic source may be used in 
conjunction with two or three spaced microphones. Phase and 
amplitude differences between these detectors will provide a 
measure of the observer's head position. Head trackers 
employing three magnetic coils to determine 6 variables of 
motion (3 position and 3 angles) are commercially available. 

The applications of this invention are numerous. These include 
remote handling, minimally invasive surgery, teleconferencing, 
computer aided design, visualisation and virtual reality. 
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CLAIMS 

1. Apparatus for the display of autostereoscopic images , 
comprising means for detecting the location of the head 
of an observer with respect to a real and/or virtual 
image forming region; optical means for providing a point 
or region of concentration of light, said light providing 
in use an image of at least a first single perspective of 
a scene at said image forming region, and a void point or 
region, spaced from said point or region of 

concentration, substantially void of light from said 
first single perspective image; means for providing an 
output signal from the detecting means indicative of the 
observer's head location; and control means responsive, 
in use, to said signal and causing the location of the 
point or region of concentration of light from said first 
single perspective to substantially track and be co- 
located with a first eye of the observer, and said void 
point or region to be co-located with the second eye of 
the observer* 

2* Apparatus as claimed in Claim 1, wherein the optical 
means comprises, in combination, a source of light; a 
lenticular screen having a substantially evenly spaced 
plurality of juxtaposed lens elements; an apertured 
surface having a corresponding plurality of light 
transmitting regions separated by opaque regions and 
positioned close to the lenticular screen between said 
screen and said source of light; and means for moving the 
apertured surface relative to said screen, whereby, in 
use, the control means positions the point or region of 
concentration of light corresponding to the perspective 
image provided by the optical means • 

3* Apparatus as claimed in Claim 2, including diffusing 
means at the focal plane of the lens elements of the 
lenticular screen * 
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4. Apparatus as claimed ? n Claim 2 or Claim 3, wherein the 
apertured surface is positioned near the focal plane of 
the lens elements of the lenticular screen; and lateral 
movement of the point or region of concentration of light 
is provided, in use, by movement in a first direction of 
said screen relative to said apertured surface by the 
means for moving, said first direction being parallel to 
the plane of said screen, 

5. Apparatus as claimed in Claim 4, wherein a change of the 
distance from the optical means of the point or region of 
concentration of light is provided, in use, by movement 
in a second direction of the screen relative to the 
apertured surface by the means for moving, said second 
direction lying in a plane orthogonal to the first 
direction of movement, 

6. Apparatus as claimed in Claim 5, wherein the optical 
means comprises a tapered structure in which the pitch 
between adjacent lens elements of the lenticular screen 
reduces in a direction orthogonal to the first direction 
of movement; the transmitting and opaque regions of the 
apertured surface are arranged in a matching tapered 
pattern; and the second direction of movement is 
substantially parallel to the plane of the screen, so 
that, in use, the pitch of the tapered screen, at a given 
position, relative to the pitch of the nearest portion of 
said pattern, is altered, whereby the control means 
controls the convergence of the light of the first 
perspective image and the distance of the point or region 
of concentration thereof from the optical means, 

7. Apparatus as claimed in Claim 6, wherein the apertured 
surface is positioned substantially at the focal plane of 
the lens elements of the lenticular screen. 
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Apparatus as Claimed , in any one of Claims 2 to 7, 
including a second optical means, which comprises means 
for providing a second point or region of concentration 
of light, said light providing, in use, an image of a 
second single perspective of the scene at the image 
forming region and a second void point or region, spaced 
from said second point or region of concentration, 
substantially void of light from said second single 
perspective image; a semitransparent mirror positioned so 
that, in use, said second perspective image is 
substantially co-located with the first perspective 
image, at least one of said first and said second 
perspective images being a virtual image; and means for 
controlling said second optical means so that, in use, 
said point or region substantially void of light from 
said second single perspective image is co-located with 
the observer's first eye and said point or region of 
concentration of light of said second perspective image 
is co-located with the observer's second eye. 

Apparatus as claimed in Claim 8, including a first 
transparent image generating surface positioned in the 
optical path between the means for providing the point or 
region of concentration for the first single perspective 
image and the observer; a second transparent image 
generating surface positioned in the optical path between 
the means for providing the second point or region of 
concentration and the observer; and means for providing, 
in use, by operation of both said first and second image 
generating surfaces, said first and second perspective 
images respectively and substantially simultaneously. 



10. Apparatus as claimed in Claim 9, wherein each of the 
image generating surfaces comprises an LCD panel. 

11. Apparatus as claimed in Claim 6, including projection 
means for providing the first and a second single 
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perspective image of ^ scene, arranged so that, in use, 
each said image is projected substantially in positional 
register with respect to the other and at respectively 
different angles onto the apertured surface so that, in 
use, the components of said first image transmitted by 
the apertured surface are projected to positions spaced 
from the corresponding components of said second image on 
the diffusing means. 

12. Apparatus as claimed in Claim 1, wherein the optical 
means comprises, in combination, a lenticular screen 
having a substantially evenly spaced plurality of 
juxtaposed lens elements; an image generating surface 
having a corresponding first plurality of spaced columns 
for providing elongate components of the first 
perspective image of the scene and a second plurality of 
columns, interleaved with said first plurality, for 
providing elongate components of a second perspective 
image of said scene, said surface or an image thereof 
being located at the focal plane of the lens elements of 
the lenticular screen, said screen providing, in use, a 
second point or region of concentration of light, 
corresponding to said second single perspective image of 
the scene and a second void point or region, spaced from 
said second point or region of concentration, 
substantially void of light from said second single 
perspective image; and means for moving, in its own 
plane, said image generating surface or an image thereof 
relative to said screen, whereby, in use, the control 
means positions the point or region of concentration of 
light corresponding to each respective single perspective 
image provided by said image generating surface. 

13. Apparatus as claimed in Claim 12, wherein the means for 
providing movement comprises means for providing a first 
direction of relative movement, orthogonal to the 
elongate image components, to control the lateral 
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position of each poi~>t or region of concentration of 
light; and means for providing a second direction of 
relative movement, orthogonal to said first direction of 
relative movement, to control the convergence of the 
image light leaving the lenticular screen and thereby the 
distance of each point or region of concentration of 
image light from said screen. 

14. Apparatus as claimed in Claim 13, wherein the optical 
means comprises a tapered structure in which the pitch 
between adjacent lens elements of the lenticular screen 
reduces in a direction orthogonal to the first direction 
of movement; the columns or, in use, an image thereof for 
providing the elongate components of each perspective 
image are arranged in a matching tapered pattern; and the 
second direction of movement is substantially parallel to 
the plane of said screen, so that, in use, the pitch of 
the tapered screen, at a given position, relative to the 
pitch of the nearest portion of said pattern, is altered, 
whereby the control means controls the convergence of the 
light of each perspective image and the distance of the 
point or region of concentration thereof from the optical 
means • 

15. Apparatus as claimed in Claim 1, wherein the optical 
means comprises a single image generating surface 
positioned in the image forming region; means for 
selectively providing by transmission or generation light 
at different positions on an aperture plane spaced from 
said image generating surface; convergent optical means 
controlled by the control means, in use, to provide 
convergence of image light from the image forming region, 
so that said aperture plane is substantially conjugate to 
a plane containing the location of the observer; means 
for alternately generating the first single perspective 
of a scene and a second perspective thereof with a second 
point or region of concentration of light corresponding 
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thereto; and means for the control means to control the 
light providing means, whereby, in use and in sequence, 
the points or regions of concentration of light 
corresponding to said first and said second single 
perspective images are provided to.be co-located with 
each respective eye of the observer, 

16. Apparatus as claimed in Claim 1, wherein the optical 
means comprises a single image generating surface 
positioned in the image forming region; area programmable 
means for providing by transmission or generation light 
at a plurality of discrete locations on a plane spaced 
from said image generating surface; and means for the 
control means to change the point or region of 
concentration of image light corresponding to a 
perspective image from the image forming region, by 
controlling, in use, the pattern of transmission or 
generation of light by said programmable means to 
correspond to spaced portions of both the first and a 
second perspective image displayed simultaneously on the 
image generating surface, so that, in use, each of the 
observer's eyes only sees the respectively correct 
perspective image portions. 



17. Apparatus as claimed in Claim 16, wherein, in use, the 
separation between light transmitting or generating 
regions of the programmable means and the corresponding 
separation between portions of the same perspective image 
on the image generating surface are decreased by the 
control means as the observer's distance from the image 
forming region, as indicated by the output signal from 
the detecting means, is increased. 



18. Apparatus as claimed in Claim 16 or Claim 17, wherein the 
positions of the light transmitting or generating regions 
of the programmable means are selected, in use, over a 
time interval, so that the observer sees a contiguous 



WO 94/20875 



PCT/GB94/00405 



three-dimensional ima^e comprising light substantially 
from the whole area of said programmable means. 

19* Apparatus for detecting the location of a marker and/or 
the head of an observer, comprising a source of infrared 
radiation; means for imaging the space illuminated by 
said source; and means for locating, in use, within the 
image of said space, the image of the marker and/or the 
head and deriving positional data corresponding thereto. 

20. Apparatus as claimed in Claim 1, wherein the means for 
detecting the location of the head of an observer 
comprises a source of infrared radiation; means for 
imaging the space illuminated by said source; and means 
for locating, in use, within the image of said space, the 
image of a marker and/or the head and deriving positional 
data corresponding thereto. 

21. Apparatus as claimed in Claim 19 or Claim 20, wherein the 
infrared source is located close to the imaging means 
and, in use, the marker comprises retro-reflective means 
exposed in use to the infrared radiation. 

22. Apparatus as claimed in Claim 21, wherein, in use, the 
marker comprises a component of the observer's headwear. 

23. Apparatus as claimed in Claim 21 or Claim 22, wherein the 
marker comprises one contiguous retro-reflective area. 

24. Apparatus as claimed in Claim 21 or Claim 22, wherein the 
marker comprises two or more retro-reflective components 
arranged in a defined pattern. 

25. Apparatus as claimed in Claim 19 or Claim 20, wherein the 
imaging means comprises a lens and a CCD array. 
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26. Apparatus as claimed .in Claim 25, wherein the CCD array 
is two-dimensional and, in use, provides video data. 

27. Apparatus as claimed in Claim 26 wherein each line of 
video data is integrated to provide positional data along 
a first axis. 

28. Apparatus as claimed in any one of Claims 25 to 27, 
wherein the infrared source is modulated in synchronism 
with the frame rate of the CCD so that, in use, an output 
signal is derived from the difference between video data 
in alternate fields. 

29. Apparatus as claimed in any one of Claims 26 to 28, 
wherein the line scan direction of the video image is 
arranged to be vertical. 



30. Apparatus as claimed in any of Claims 26 to 29, wherein, 
in use, one or more lines of video data are selected to 
comprise data corresponding to the retro-reflective 
means, whereby a second axis of positional data and/or 
orientation data is derived. 

31. Apparatus as claimed in any one of Claims 19 to 30 
wherein, in use, the width of the image of the retro- 
reflective means is determined, whereby its distance from 
the detecting means, and thus the location of the 
observer in three dimensions, is derived. 

32. Apparatus as claimed in any one of Claims 19 to 31, 
including means for interpolating between intensity 
values associated with two or more discrete locations on 
the CCD array. 



33. Apparatus as claimed in any one of Claims 19 to 32, 
including a blocking filter to remove light outside the 
range of wavelengths of interest. 
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34. Apparatus as claimed in Claim 1 or Claim 19, wherein the 
first perspective image and/or a second perspective image 
are selected in accordance with the position of the 
observer as provided by the output signal from the 
detecting means. 

35. Apparatus as claimed in Claim 34, including means for 
moving the pupil of a camera for providing an image for 
viewing by the observer; and means for causing said pupil 
moving means to be responsive, in use, to changes in the 
output signal and/or positional data therefrom, whereby 
movement parallax is provided* 

36. Apparatus as claimed in Claim 19 including a field 
sequential viewing device, wherein the imaging means 
comprises a CCD array for providing video data; the 
infrared source is modulated, in use, and the frame rate 
of the CCD and the operation of said field sequential 
viewing device are synchronised to the period of the 
modulation of said source. 

37. A method for displaying autostereoscopic images, 
comprising detecting the location of the head of an 
observer with respect to a real and/or virtual image 
forming region; providing a point or region of 
concentration of light from an image of at least a first 
single perspective of a scene at said image forming 
region, and a void point or region, spaced from said 
point or region of concentration, substantially void of 
light from said first single perspective image; providing 
an output signal indicative of said observer's head 
location; and causing, responsive to said signal, the 
location of said point or region of concentration of 
light from said first single perspective to substantially 
track and be co-located with a first eye of said observer 
and said void point or region to be co-located with the 
second eye of said observer. 
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38. A method as claimed in Claim 37, wherein providing the 
point or region of concentration of light comprises 
providing a source of light; providing a lenticular 
screen having a substantially evenly spaced plurality of 
juxtaposed lens elements; positioning an apertured 
surface having a corresponding plurality of light 
transmitting regions separated by opaque regions close to 
said lenticular screen between said screen and said 
source of light; and moving the apertured surface 
relative to said screen to locate the point or region of 
concentration . 

39. A method as claimed in Claim 37, wherein providing the 
point or region of concentration cf light comprises 
positioning a single image generating surface in the 
image forming region; providing light by transmission or 
generation at a plurality of discrete locations in a 
plane spaced from said image generating surface ; 
controlling the pattern of transmission or generation of 
said light; and controlling a pattern of spaced portions 
of both the first and a second perspective image 
simultaneously on the image generating means to 
correspond to said pattern of transmission or generation 
of light, so that each of said observer's eyes only sees 
the respectively correct perspective image portions. 

40. A method for detecting the location of a marker and/or 
the head of an observer, comprising providing a source of 
infrared radiation; imaging the space illuminated by said 
source; locating, in use, within the image of said space, 
the image of the marker and deriving positional data 
corresponding thereto . 



43 



BN8DCXHD: <WO_9420675A2JL?- 



WO 94/20875 



PCT/GB94/00405 




1/12 



BNSOOCID: <WO_^M20e75A2JU» 



WO 94/20875 PCT/GB94/00405 




2 / 12 



BN80OCJD: <WO__£42067CA2JL> 



WO 94/20875 PCT7GB94/00405 




BN8DOCID: <WO_M20e75Aa_U» 



WO 94/20875 PCT/GB94/00405 




4/12 



BN800CCD: <WO_j94a087SAaJU* 



WO 94/20875 PCT/GB94/0040S 




BN80OCID: <W Q 0 42087542 I > 



i 



WO 94/20875 



PCT/GB94/00405 




6/12 



BN8DOC1D: <WO_jM2O075A2JL> 



WO 94/20875 



PCT7GB94/00405 




BN8000D: <WO_^M2067SA2JU» 



WO 94/20875 



PCT/GB94/00405 




8/12 



BHSDOCIO: <WO_&420875A2JU> 



WO 94/20875 



PCT/GB94/00405 




BN8DOCID: <WO_jM20675A2JL> 



WO 94A20875 PCT/GB94/00405 




BN8DOCID: <WO_9420875A2JU> 




BN8DOaD: <WO_W20675A2JU> 



WO 94/20875 



PCT/GB94/00405 



m 




CD 



7 



12/12 



BNSDOCfO. <WO_9420875A2JL^ 



This Page Blank (us 



